Shimmer correction is one of image correction approaches used for a surveillance camera system. The “shimmer” in this case refers to the phenomenon of local shimmer of the image owing to fluctuation in the refractive index inside the optical path, which is caused by the thermal turbulence and the like. This phenomenon is referred to as the heat haze, causing deterioration in visibility. Correction of the shimmer may improve visibility.
Several kinds of techniques for correcting the shimmer in the image have been proposed. As one of the aforementioned techniques, correction of the deformed image caused by the shimmer, which is acquired in the form of the motion vector has been proposed. This technique, however, requires the local shimmer by pixel, causing the problem of a significantly large computation amount.
There has been proposed the technique of alleviating the influence of shimmer by smoothing the time-series image (video image) in a direction of time (see, for example, Patent Literature 1 and Non-Patent Literature 1). The shimmer may irregularly displace the pixel position depending on time. The displacement is in a manner of normal distribution with respect to the center point. Therefore, a plurality of pixels at different time points are simply overlapped with one another for smoothing so as to alleviate the influence of the shimmer, providing the original configuration of the subject.
Correction of the shimmer through time smoothing is highly effective for the immobile subject. However, in the case where the moving object exists in the image, the afterimage will be left in the region through which the moving object has passed, resulting in a smeared outline.
FIG. 5 schematically shows a generally employed shimmer correction technique through the time smoothing process. Codes 171, 172, 173 in the time-series input images 11, 12, 13 denote images of a subject (tree) 17, which are influenced by shimmer. Codes 181, 182, 183 in the input images 11, 12, 13 denote images of a subject (pillar) 18, which are influenced by shimmer. Codes 191, 192, 193 in the input images 11, 12, 13 denote images of a subject (vehicle) 19, which are influenced by shimmer. In the case where the input images 11, 12, 13 are under the influence of shimmer, visibility of the subject image is deteriorated.
In order to alleviate the influence, an attempt of time smoothing pixel values of a plurality of temporally proximate frames has been made. An image 54 denotes the one corrected by time smoothing. A section 554 in the image 54 denotes the corrected image of the subject (tree) 17. A section 564 in the image 54 denotes the corrected image of the subject (pillar) 18. In this case, the correction is effective for the static subjects 17, 18 in the image 54 corrected through time averaging. In a region 58 where the moving object 19 has existed, smear owing to motion blur is observed, which may deteriorate the image quality. The time smoothing process will be defined as filtering executed in the region including a plurality of pixels in time direction through the uniform method. In this case, the term “uniform” means that the shimmer is not considered at the pixel level. Accordingly, the “uniform” applies to the smoothing process which involves affine transformation (to be described later) evenly performed over the region in accordance with deformation of the moving body.
Meanwhile, the applicant of the present invention filed the application, Patent Application No. 2012-107470 (which will be referred to as earlier application) describing the technique that suppresses smear of the image by calculating similarity between the correction-targeted image and the time smoothed image, and accurately recognizing the region of the moving body for outputting without executing the time smoothing process. The technology relevant to the present invention has been filed with Patent Application No. 2013-57774 (which will be referred to as related invention).